1. Field of the Invention
The present invention relates generally to image/video display and processing systems, and in particular, to a method, apparatus, and article of manufacture for storing and accessing images in a real-time compositing system.
2. Description of the Related Art
Audio and video files, in the form of a plurality of digitized frames or clips, are often very large and consume considerable time and bandwidth to display and process. During the editing process, such frames/clips are often combined or composited together to provide an end result. In addition, when previewing and working with such clips, the data needs to be moved from disk storage to memory to the central processing unit (CPU). To achieve the highest quality results, image compositing systems/applications need to be real-time, allowing authors/editors to preview clips (as closely as possible) at the quality at which they will be finally rendered. However, the large size and amount of such data prevents prior art applications from achieving real-time compositing performance for clips above a certain resolution (e.g., 2K).
The result of the hardware limitations forces editors to preview edits using an approximation of the final render, which typically uses lower resolution versions of the media/data as a proxy. This problem appears in compositing systems for clips that require large background plates over which the camera will pan. As a result, editors must make critical artistic (and business) judgments based on this lower-quality version. Alternatively, the editors may take the time to perform a final render on a clip. Such a final rendering operation consumes additional time and incurs additional costs. Accordingly, what is needed is a tool/methodology for enabling real-time performance of a compositing system. These problems may be better understood with a further explanation of prior art compositing systems and storage operations.
As described above, prior art compositing systems often utilize lower quality images during editing/compositing operations. In this regard, the prior art systems failed to provide any real-time compositing with images larger than a certain size. For example, prior art compositing may have performed real time compositing with images that fit on a single texture using texture memory. However, such processing was not performed on larger size images and thus restricted the operations (and the accuracy of such operations) that could be performed.
In addition to prior art compositing operations, the prior art utilizes various schemes to store image data in memory or on disk. For example, disk-based tiling for storage and manipulation of large images is common in the prior art. For example, numerous image-processing systems store images as collections of fixed-size tiles on disk and retrieve only the required tiles into memory on an as-needed basis. However, disk-based tiling along cannot enable a compositing system to achieve real-time performance. For example, the size of the fixed tiles used may not be optimized for disk input/output (I/O), graphics I/O, and/or CPU processing.
To expedite processing, some prior art systems adopted their own proprietary file system wherein space was reserved and allocation could be controlled. However, such systems prevented the ability to utilize open file systems such as NTFS™ (NT File System—utilized in Windows™) or XFS™ (X File System—utilized in Linux™). Such open file systems split files into blocks and do not guarantee a contiguous file.
Testing of such open file systems indicate that files larger than a certain size (e.g., 8 MB) are likely not to be contiguous. To expedite processing, it is desirable to minimize the disk I/O. Accordingly, an image may be split into tiles based on the hardware size testing (e.g., tiles of 8 MB). However, for processing, the graphics I/O (e.g., the video processing unit) or the CPU may prefer smaller tiles. Accordingly, the larger size tiles would minimize disk I/O but complicate and extend the time to perform any further processing. Further, it may not be possible to perform any processing using such tiles (i.e., if the video processing unit [VPU] does not or cannot handle a tile of such a large size).